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Abstract 

Upon completion of a graduate level course at the Open University of Israel, one instructor 
received very high student ratings while the other received very low ratings. We utilized this 
exceptional situation to perform ad hoc analyses of their course forums. The objective of this 
study was to map the dialogic behavior that occurred and to create suggestions for best practice 
and for worst practice in terms of active and passive participation, instructor response time, and 
the extent of teaching presence, social presence, and cognitive presence. 
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Introduction 

Teaching effectiveness may be defined as how an instructor can best direct, facilitate, and support 
students toward certain academic ends, such as achievement and satisfaction. Teaching 
effectiveness has been investigated extensively in traditional classrooms for more than seven 
decades (for a meta-analysis of empirical studies from 1995-2004, see Seidel & Shavelson, 2007). 
Over the past five years, research has become directed toward teaching effectiveness in online or 
virtual classes. As a preface to our study, we discuss findings and conclusions concerning 
teaching effectiveness in traditional classrooms. We do so for two reasons: practically, since all 
findings are relevant to online teaching, and theoretically, since current research has shown the 
theoretical equivalency of all kinds of instructional systems within the framework of a unified 
theory of instructional design (Gorsky & Caspi, 2005; Gorsky, Caspi, & Chajut, 2007). 

One of the most widely cited sources for teacher effectiveness in traditional classrooms is 
Chickering and Gamson (1987), who suggested seven principles for good practice in 
undergraduate education. Given their simplicity and eloquence, no further commentary is added. 
A good teacher does the following: 

1 . encourages student-faculty contact. 
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2. encourages cooperation among students, 

3. encourages active learning, 

4. gives prompt feedback, 

5. emphasizes time on task, 

6. communicates high expectations, 

7. respects diverse talents and ways of learning. 

The authors added this statement: 

These principles are intended as guidelines for faculty members, students, and 
administrators... to improve teaching and learning. They rest on 50 years of 
research on the way teachers teach and students learn, how students work . . . with 
one another, and how students and faculty talk to each other. (Chickering & 
Gamson, 1987, p.3) 

Given their importance for guiding teaching in traditional classrooms, these principles have since 
been adapted to web-based and virtual classrooms that rely on diverse instructional technologies. 
A decade after the principles were published, Chickering and Ehrmann (1996) wrote the 
following: 

Since the seven principles of good practice were created in 1987, new 
communication and information technologies have become major resources for 
teaching and learning in higher education. If the power of the new technologies is 
to be fully realized, they should be employed in ways consistent with the seven 
principles. (Chickering & Ehrmann, 1996, p.2) 

These seven principles are defined formally. How can we define these principles operationally so 
as to be appropriate for research in online classrooms? We propose, as have others (i.e., Shea, 
Pickett, & Pelz, 2003), that the community of inquiry model (Garrison, Anderson, & Archer, 
2000) reflects the principles of good practice in undergraduate education and can accurately 
quantify them. Relying on a quantitative content analysis technique, the model affords a detailed 
set of categories and indicators through which to examine issues of pedagogy, dialogue, and 
interaction. We now present an overview of the model along with representative research. 

The Community of Inquiry Model 

Garrison, Anderson, and Archer (2000) developed the community of inquiry model as an online 
learning research tool. The framework consists of three elements - cognitive presence, teaching 
presence, and social presence - as well as categories and indicators to define each of the 
presences and to guide the coding of transcripts. Cognitive presence is defined by Garrison, 
Anderson, and Archer (2001) as the extent to which participants are able to construct meaning 
through sustained communication. Teaching presence includes subject matter expertise, the 
design and management of learning, and the facilitation of active learning (Anderson, Rourke, 
Garrison, & Archer, 2001). Social presence is the perceived presence of others in mediated 
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communication (Rourke, Garrison, & Archer, 1999), which Garrison et al. (2000) contend 
supports both cognitive and teaching presence through its ability to instigate, to sustain, and to 
support interaction. It had its genesis in the work of John Dewey and is consistent with all 
theoretical approaches to learning in higher education. This framework has provided significant 
insights and methodological solutions for studying online learning (Garrison & Archer, 2003; 
Garrison, Cleveland-Innes, Koole, & Kappelman, 2006). The structure of the community of 
inquiry model has been confirmed through factor analysis (Arbaugh & Hwang, 2006; Garrison, 
Cleveland-Innes, & Fung, 2004; Shea & Bidjerano, 2009). 

Social presence is described as the ability to project one’s self and to establish personal and 
puiposeful relationships (Rourke et al., 1999). The three main categories of social presence are 
affective communication, open communication, and group cohesion. Richardson and Swan 
(2003) explored perceptions of social presence in online courses and found that students’ 
perceptions of social presence were highly correlated with perceived learning and satisfaction 
with their instructors (see also Steinweg, Trujillo, Jeffs, & Hopfengardner- Warren, 2006). 
Picciano (2002) found relationships between student perceptions of social presence, learning, and 
interactions in the course discussions. The positive correlation between perceived social presence, 
seen according to the community of inquiry model as self projection, and most aspects of 
perceived learning may lead to the conclusion that social presence affords learning by setting a 
convenient climate (Caspi & Blau, 2008). However, actual interaction in the course discussions in 
Picciano's (2002) study was not correlated with actual performance (their scores on a multiple 
choice exam and on a written assignment). Whether and how actual social interaction might or 
might not affect actual learning online remains unclear and constitutes an important area for 
future research (Caspi & Blau, 2008; Swan & Shea, 2005). Swan (2002) studied the apparent shift 
of social presence over time in online course discussions. She reported that open communication 
indicators (“affective” and “interactive”) of social presence increased over time, while cohesive 
indicators decreased. One possible explanation is that the use of such references became less 
necessary as a galvanized classroom community was formed. Another possible explanation 
addressed the fact that discussion was more exploratory than collaborative. Contrary to the nature 
of the shift in social presence reported by Swan (2002), Vaughan (2004) and Vaughan and 
Garrison (2006) found that the frequency of affective and open communication comments 
decreased, while group cohesion comments increased. It is important to note that the context of 
Vaughan’s study (2004) was a blended professional development community. The interpretation 
was that affective and open communication was necessary to establish a sense of community. It 
was only after the social relationships were established and the group became more focused on 
purposeful activities that cohesive comments began to take precedence. Social presence online 
becomes somewhat transparent as the focus shifts to academic puiposes and activities. 

Cognitive presence is defined as the exploration, construction, resolution, and confirmation of 
understanding through collaboration and reflection in a community of inquiry (Garrison et al., 
2001). Cognitive presence is grounded in the work of Dewey (1933) on reflective thinking. Four 
phases are defined: Triggering event, exploration, integration, and resolution. Garrison et al. 
(2001) argue that the third phase, integration, is the most difficult to detect from a teaching or 
research perspective. This phase requires active teaching presence to diagnose misconceptions, to 
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provide probing questions, comments, and additional information in an effort to ensure 
continuing cognitive development, and to model the critical thinking process. Often students will 
be more comfortable remaining in a continuous exploration mode; therefore, teaching presence is 
essential in moving the process to more advanced stages of critical thinking and cognitive 
development. 

Teaching presence is defined as “the design, facilitation and direction of cognitive and social 
processes for the purpose of realizing [students’] personally meaningful and educationally 
worthwhile outcomes” (Anderson et ah, 2001, p.5). Vygotsky’s (1978) scaffolding analogies 
illustrate an assistive role for teachers in providing instructional support to students from their 
position of greater content knowledge. Although many authors recommend a “guide on the side” 
approach to moderating student discussions, a key feature of this social cognition model is the 
adult, the expert, or the more skilled peer who scaffolds a novice’s learning (Anderson et ah, 
2001). The community of inquiry model defines three categories of teaching presence: design and 
organization, facilitating discourse, and direct instruction. The categories of teacher presence 
have been tested by Anderson et al. (2001) in an analysis of the complete transcripts of two online 
courses and have proved reasonably reliable and useful in identifying differences in both the 
quantity and quality of the teaching presence projected by different online instructors. How these 
differences might relate to community has not yet been hypothesized, but the community of 
inquiry model might provide a starting point for such investigations (Swan & Shea, 2005). 

The body of evidence attesting to the importance of teaching presence for successful online 
learning is growing rapidly (Garrison & Cleveland-Innes, 2005; Meyer, 2003; Murphy, 2004; 
Pawan, Paulus, Yalcin, & Chang, 2003; Shea, Pickett, & Pelz , 2004; Swan, 2002; Swan & Shih, 
2005; Varnhagen, Wilson, Krupa, Kasprzak, & Hunting, 2005; Vaughan, 2004; Wu & Hiltz, 
2004). The consensus is that teaching presence is a significant determinate of perceived learning, 
student satisfaction, and sense of community. Perceived teaching had a strong direct effect on 
self-reported learning outcomes (LaPointe & Gunawardena, 2004). Each category of a tutor’s 
presence is vital to learning and to the establishment of the learning community; tutors' behavior 
must be such that they are seen to be “posting regularly, responding in a timely manner and 
modeling good online communication and interaction” (Palloff & Pratt, 2003, p.l 18). Without an 
instructor’s explicit guidance and “teaching presence,” students were found to engage primarily 
in “serial monologues” (Pawan et al., 2003). Baker (2004) discovered that “instructor immediacy, 
i.e., teaching presence (Rourke et al., 1999), was a more reliable predictor of effective cognitive 
learning than whether students felt close to each other. Studies have demonstrated that instructor 
participation in threaded discussion is critical to the development of social presence (Shea, Li, 
Swan, & Pickett, 2005; Swan & Shih, 2005) and sometimes not fully appreciated by online 
faculty (Liu, Bonk, Magjuka, Lee, & Su, 2005). Shea, Li, and Pickett (2006) proposed that 
teaching presence - viewed as the core role of the online instructor - is a promising mechanism 
for developing learning community in online environments. The majority of students and 
instructors in Vesely, Bloom, and Sherlock’s (2007) study identified the same elements for 
building online community, but students ranked instructor modeling as the most important 
element in building online community, while instructors ranked it fourth. 
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To conclude, we note a recent study that questioned whether there are three categories of teaching 
presence. Shea (2006), who completed an extensive study of teaching presence and online 
learning, concluded that two categories (“design” and “directed facilitation”) sufficed to define 
the construct. In this study, we opt for the three categories of teaching presence as defined 
originally by Garrison et al. (2000). 

Extensive research into effective online teaching has been carried out without using the 
community of inquiry model. We now present representative findings concerning the impact of 
interaction and feedback on students’ perceptions of, and satisfaction with, online learning. Both 
constructs are closely interrelated with instructor response time, the rate at which instructors 
respond to students’ posts. 

Effective Online Teaching: Research Findings 

Regarding interaction, Ridings and Gefen (2004) found that across all types of communities, 
information exchange was the most commonly cited reason for participation while social support 
was the second most popular reason for members in communities devoted to professional topics. 
Communities are used not only for informational purposes but also as an opportunity for social 
interaction (Kaye, 2005). Kalman, Ravid, Raban, and Rafaeli (2006) argued that interactivity is 
an essential characteristic of effective online communication and plays an important role in 
keeping message threads and their authors together. Interactive communication (online as well as 
in traditional settings) is engaging, and loss of interactivity results in a breakdown of the 
communicative process. 

Research indicates the existence of a relationship between learners’ perceptions of social presence 
and their motivation for participation in online discussions (Weaver & Albion, 2005). The 
importance of social interaction to individuals who participate in online communities explains 
why sociability may be a key element in determining the success or failure of an online 
community. It has been found that learner satisfaction depends on student-instructor interactions 
and that students’ perceptions of ‘good’ interactions have a positive impact on their enthusiasm 
and learning (Swan, 2001; Tricker et al., 2001; Ussher, 2004). 

Qualitative data from Weaver and Albion’s (2005) study showed that students placed a high 
priority on the role of the course instructor as initiator and maintainer of momentum in 
discussions. This was most evident in the comments of students who experienced infrequent 
participation by the course instructor. Northrup (2002) found that online learners felt it was 
important for instructors to promote collaboration and conversation. When interactive activities 
are carefully planned, they lead not only to greater learning but also to enhanced motivation 
(Berge 1999; Northrup, 2002). 

Interaction in a forum is closely related to the rate at which new messages are posted. Increased 
frequency of posting in asynchronous communications can lead to more favorable impressions of 
communication partners (Liu, Ginther, & Zellhart, 2001; Walther & Bunz, 2005). Researchers 
have suggested that timing of messages can serve as a proxy for a sense of social presence 
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(Blanchard, 2004), as an indication of attentiveness (Walther & Bunz, 2005) or respect (Bargh & 
McKenna, 2004), and as a clue to the sociability of a community (Maloney -Krichmar & Preece, 
2005). As such, the frequency of messages may serve as a signal for how engaged participants are 
with the community. 

Regarding feedback, a survey conducted by McCollum, Calder, Ashby, and Morgan (1995) 
showed that students ranked feedback as the highest factor in determining course quality. Similar 
findings were reported by Tricker et al. (2001), Spangle, Hodne, and Schierling (2002) and 
Young (2006). At the same time, faculty members found interacting with and providing feedback 
to students in online classes to be more time consuming than in face-to-face classes (Chabon, 
Cain, & Lee-Wilkerson, 2001; Jennings & McCuller, 2004; Herrmann & Popyack, 2003; Smith, 
Ferguson, & Caris, 2002). 

Kearsley (2000) and Simonson, Smaldino, Albright, and Zvacek (2000) found that instructors can 
either enhance or decrease course interaction depending on how consistently, quickly, and 
helpfully they respond. The most successful online courses are those in which instructor -to- 
student interaction is both frequent and productive (Northrup, 2002; Swan, 2001). Thus, evidence 
of high instructor engagement ranged from low to high in the quality, speed, and usefulness of 
feedback to students. Jiang and Ting (2000) further reported that both perceived learning and 
perceived interaction with instructors were linked to the actual average numbers of responses per 
student that instructors made. Swan, Shea, Fredericksen, Pickett, Pelz, and Maher (2000) also 
found a correlation between students’ perceived interaction with their instructors and the actual 
frequency of instructor participation in online course discussions, and Picciano (1998) reported 
that instructors’ activity was related to students’ perceived learning from them in an online 
graduate level course. The number and type of facilitator postings also increased the level of 
interaction between students (Gilbert & Dabbagh, 2005). 

On the other hand, Eom (2006) found that instructor facilitation and feedback did not affect the 
perceived satisfaction of students who take web-based courses. Eom found that the most 
significant factors for increasing student satisfaction with online classes are paying attention to 
students and responding to their concerns. 

The Current Study: Rationale and Research Questions 

At the end of an ordinary graduate level course at the Open University of Israel, one instructor 
received very high student ratings while the other received very low ratings, relative to university 
standards. Given this exceptional situation, we used the quantitative content analysis technique 
(Garrison et al., 2000) and data logs to perform post-hoc analyses of their forums. The 
quantitative content analysis technique, based on the community of inquiry model, has been 
widely used to analyze forums (Garrison, 2007). Given the reliability and validity of this 
procedure and that all other relevant variables in the learning environment (course policy, content 
and difficulty, equivalent numbers of instructor assignments, group size, semi -random assignment 
to groups) were controlled, we expected to identify the impact of the instructors’ actions on the 
dialogic behavior in the two forums. 
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We defined two objectives for this study. The first was to test the hypotheses, theory-based and 
empirically supported, that in a forum led by a very highly rated instructor, as opposed to a forum 
led by an instructor held in very low esteem, the following would occur: 

1 . Active and passive participation would be significantly higher, 

2. Levels of social presence, teaching presence, and cognitive presence would be 
significantly higher, and 

3. Instructor response time would be significantly shorter. 

The second was to create initial suggestions for comparing one asynchronous, virtual learning 
community with another. Specifically, we hoped to establish tentative boundary conditions for 
best practice and for worst practice in terms of active and passive participation in the forum, 
instructor response time, the extent of social presence, teaching presence, and cognitive presence, 
and the ratios of each. 


Methodology 


Background 

The Open University of Israel is a distance education university that offers undergraduate and 
graduate studies to students throughout Israel. The learning environment is blended: The 
University offers a home study system based on textbooks, face-to-face tutorials, and a web- 
based instructional environment wherein each course has its own website. Course sites simplify 
organizational procedures and enrich students’ learning opportunities and experiences. Website 
use is optional or non-mandatory so that equality among students is preserved. It does not replace 
textbooks or face-to-face tutorials, which are the pedagogical foundations of the Open University. 
The website provides forums for asynchronous instructor-student and student-student 
interactions. Each course has a coordinator who is responsible for all administrative and academic 
activities and instructors who lead tutorials. Instructors and coordinators are available for 
telephone consultations at specified days and times. Course coordinators define the number of 
forums made available and their purpose. 

In this particular course, three forums were defined: a central forum, led by the course 
coordinator and available to all students, which dealt with administrative and organizational 
issues and two smaller forums, led by each of the instructors, which dealt with subject-matter 
issues. Prior to the start of the course, the coordinator met with both instructors and defined for 
them the goal of their forums: to provide a haven wherein subject-matter issues could be 
criticized, reflected upon, discussed, and debated within the context of the course, beyond the 
context of the course (students’ personal experience), and in any other way deemed appropriate. 
Both instructors were expressly told to encourage and to facilitate such activities. It appeared that 
both accepted the task readily and eagerly. In addition, at the start of the course, the course 
coordinator communicated to all the students (via the general course forum) the purpose of the 
instructor-led forums. As well, each instructor clearly communicated to her group of students the 
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goals and expectations she held vis-a-vis the forum. This was done by posting messages in the 
forum and at the face-to-face tutorial sessions. 


Participants 

Participants were two female instructors and 42 graduate students enrolled in the course. Of the 
42 students, thirty (71%) were female. Students were divided equally into two forums led by each 
of the two instructors. One possibly significant difference between the two instructors was their 
prior experience in tutoring Open University courses: one instructor, subsequently held in low 
esteem, had 4 years experience, while the second instructor, subsequently held in high esteem, 
was new to the task. She was, however, experienced in the ways of distance education. It is 
especially noteworthy that previous evaluations of the instructor subsequently held in low esteem 
were generally average, not below average or low. 

Instruments and Procedure 

Three instruments were employed for obtaining data: (1) survey data, elicited near the course’s 
end, rated students’ assessments of the instructors and perceived course difficulty; (2) the course 
log site recorded the amount of active participation, the amount of passive participation (lurking), 
the number of threads, and instructor response time; and (3) the quantitative content analysis 
technique, based on the community of inquiry model (Garrison et al., 2000), was used to code and 
analyze transcriptions from the two forums. This technique has been widely used; it is reliable 
and valid (Garrison, 2007). Its implementation, however, requires that several methodological 
issues be resolved. 

One issue is the level of coding (e.g., indicator vs. category). Content analysis, as described by 
Rourke and Anderson (2004), is time consuming, and coding at the indicator level is difficult, 
often yielding poor reliability (Murphy & Ciszewska-Carr, 2005). In this study, we coded at the 
category level (Garrison et al., 2006). 

Another issue is the unit of analysis. Rourke et al. (1999) identified five units of analysis used in 
computer conferencing research: proposition units, sentence units, paragraph units, thematic 
units, and message units. While there has been some discussion around this issue (Garrison et al., 
2006; Fahy, 2001; Rourke, Anderson, Garrison, & Archer, 2001), it remains a challenging 
decision influenced by research question and context. In the present study, we used the message 
unit, in accord with Anderson et al.’s (2001) study of teaching presence, Garrison et al.’s (2001) 
study of cognitive presence, and Rourke et al.’s (1999) study of social presence. 

Other issues are objectivity, reliability, and replicability (Rourke et al., 2001). No established 
standards exist for inter-rater reliability (De Wever, Schellens, Valcke, & Van Keer, 2006). There 
is no consensus for the percent agreement statistic. Often a cut-off figure of 0.75-0.80 is used to 
determine reliability; others use 0.70 (Neuendorf, 2002; Rourke et al., 2001). To increase 
reliability and to control errors brought on by inexperience or misinterpretation, Garrison et al. 
(2006) suggest a negotiated coding approach: researchers code the transcripts and then actively 
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discuss their respective codes with their fellow judges in order to achieve consensus or near 
consensus. Gros and Silva (2006) propose the use of a research methodology based on the 
intervention of the participants, especial course instructors, for analyzing computer supported 
communication. In this study we used the traditional coding approach (without negotiation or 
participant intervention): 25% of postings were randomly chosen and re-estimated by a second 
rater; 92% agreement was achieved (Cohen’s k = 0.93). 

Findings 

The Open University of Israel routinely offers students the opportunity to rate each course they 
participate in along several dimensions. Questionnaires are posted in the course forum close to 
the semester’s end, prior to the final examination. Anonymity is assured. For the course involved, 
23 of the 43 students (53.5 %) returned questionnaires. 

One dimension, instructor satisfaction, includes a cluster of 12 questions; each is rated on a scale 
of 0-5, with five indicating high satisfaction. Overall, the highly esteemed instructor received an 
average rating of 4.3 ( SD : 0.26) while the instructor held in low esteem received a rating of 2.3 
(, SD : 0.42). Figure 1 shows ratings achieved by the two instructors for each of the 12 questions. 
The relatively small numbers of students involved precluded the use of t test or Mann Whitney 
statistics. Flowever, differences between students’ satisfaction vis-a-vis the two instructors for all 
parameters were apparent. 



Figure 1. Students’ evaluations of instructors. 


We note that no specific question or questions address how instructors managed their 
asynchronous course forums. For the purpose of this research, we assumed that student 
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satisfaction vis-a-vis this parameter would be similar to all the other parameters. Indeed, the 
findings described below support this assumption. 

Data obtained from the course log site enabled measurement of active and passive participation 
for each forum. Passive participation, lurking, occurs when a student logs onto the forum but does 
not post a message. Highly significant differences were found between the two forums for active 
and passive participation by both instructors and students. Forum 1, led by the instructor held in 
low esteem, included a total of 55 postings; forum 2, led by the highly esteemed instructor, 
included a total of 125 postings (f = 27.22, p < 0.001). Table 1 summarizes these findings for the 
duration of the semester. 

Table 1 

Active and Passive Participation (Lurking) During the Semester 


Participation 

Months 

Foruml 

(low esteem) 

Forum2 

(high esteem) 

x 2 

Instructor 

postings 

1 st 

4 

6 

n.s. 

"2 s3 

11 

7 

n.s. 


4 

29 

18.94*** 

4 th 

1 

11 

8.33** 

Totals 

20 

53 

14.92*** 

Student 

postings 

1 st 

12 

12 

n.s. 

2 nd 

14 

8 

n.s. 


8 

40 

21.33*** 


1 

12 

9.31** 

Totals 

35 

72 

12 79*** 

Threads 

1 st 

5 

7 

n.s. 

"r 3 

7 

6 

n.s. 


5 

18 

7.35** 

4 th 

1 

7 

4.5* 

Totals 

18 

38 

7.14** 

Lurking 

1 st 

181 

207 

n.s. 

-^3 

276 

260 

n.s. 

~y* 

187 

487 

133.53*** 

4 th 

131 

435 

163.28*** 

Totals 

775 

1389 

174.21*** 


* p < . 05 , ** p < . 01 , *** p < .001 


Regarding active participation, data showed that while the instructor and students in forum 2 
posted until the end of semester, the instructor and students in forum 1 stopped posting 
approximately a month earlier. This may indicate a total loss of interest in the forum as a learning 
resource. Indeed, threads, which are essentially dialogues, came to a near halt after the third 
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month in the forum led by the instructor held in low esteem. Regarding passive participation, data 
indicated that students in forum 2 entered in high rates; students in forum 1, despite the lack of 
activity, continued to enter the forum until the semester’s end, possibly in the hope that 
something of consequence had happened. 

Next, the quantitative content analysis technique was applied in order to determine the extent of 
teaching, social, and cognitive presence for each forum. The first analytic procedure viewed both 
forums in their entirety; that is, all instructor and student postings were evaluated. Table 2 
presents these data. 

Table 2 

Teaching, Social, and Cognitive Presence 


Presence 

Foruml 

(low esteem) 

Forum2 

(high esteem) 

instances 

% 

instances 

% 

Teaching 

43 

24.4 

73 

19.9 

Social 

78 

44.3 

236 

64.3 

Cognitive 

55 

31.3 

58 

15.8 

Totals 

176 

100% 

367 

100% 


First, highly significant differences were found between the two forums for instances of teaching 
presence (y 2 = 7.77, p < .01) and for social presence (y 2 = 79.50, p < .001); no significant 
difference was found for cognitive presence (y 2 = 0.04, n.s.). Second, a highly significant 
difference was found between the two forum’s internal distribution of the presences expressed as 
percentages ( y 2 (2) = 23.05, p < .001). 

In order to gain deeper insights into these findings, we next investigated the teaching, social, and 
cognitive presence actively engaged in by each of the instructors. Significant differences were 
found between the two instructors for all three presences: teaching presence ( y 2 = 17.75, p < 
.001), social presence (y 2 = 64.10, p < .001) and cognitive presence (yj- 4.26, p < .05). Tables 
3, 4, and 5 summarize occurrences of each type of presence along with the related categories. 
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Table 3 

Teaching Presence: Instructors ’ Postings Only 


Categories 

Months 

Foruml 

(low esteem) 

Forum2 

(high esteem) 

x 2 

Design 

1 st 

1 

3 

n.s. 


4 

0 

n.s. 

"3 s 

0 

3 

n.s. 

4 th 

0 

0 

n.s. 

Totals 

5 

6 

n.s. 

Discourse 

1 st 

2 

3 

n.s. 

“r 3 

3 

0 

n.s. 

"3 s 

0 

13 

13.00*** 

4 th 

0 

3 

n.s. 

Totals 

5 

19 

8.17** 

Instruction 

1 st 

3 

2 

n.s. 

“2 s3 

4 

5 

n.s. 


0 

13 

13.00*** 

4 th 

0 

7 

7.00** 

Totals 

7 

27 

1 1 77*** 


** p < . 01 , *** p < .001 


Table 4 

Social Presence: Instructors ’ Postings Only 


Categories 

Months 

Foruml 

(low esteem) 

Forum2 

(high esteem) 

x 2 

Affective 

1 st 

0 

4 

n.s. 

“2 s3 

3 

3 

n.s. 


0 

19 

19.00*** 

4 th 

0 

1 

n.s. 

Totals 

3 

27 

19 2*** 

Open 

communication 

1 st 

4 

4 

n.s. 

“2 s3 

11 

7 

n.s. 

3 rd 

3 

29 

2i 13*** 

4 th 

0 

11 

11.00*** 

Totals 

18 

51 

15.78*** 

Cohesion 

1 st 

3 

6 

n.s. 

“2 s3 

4 

6 

n.s. 

"3 s 

0 

27 

27.00*** 

4 th 

0 

11 

11.00*** 
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Totals 

7 

50 

32.44*** 


**/?<. 01 , ***/?<. 001 

Table 5 


Cognitive Presence: Instructors ’ Postings Only 


Indicators 

Months 

Foruml 

(low esteem) 

Forum2 

(high esteem) 

x 2 

Trigger 

1 st 

3 

3 

n.s. 

_ 2 5a 

5 

0 

5.00* 

P 3 

0 

0 

n.s. 


0 

0 

n.s. 

Totals 

8 

3 

n.s. 

Exploration 

1 st 

1 

2 

n.s. 

_ 2 5a 

3 

2 

n.s. 

IP 

0 

3 

n.s. 

Tp 

0 

1 

n.s. 

Totals 

4 

8 

n.s. 

Integration 

1 st 

0 

2 

n.s. 

IP 

2 

1 

n.s. 

"P 3 

0 

7 

7.00** 

- 4 m 

0 

0 

n.s. 

Totals 

2 

10 

5.33* 

Resolution 

1 st 

0 

1 

n.s. 

IP 

2 

1 

n.s. 

P 3 

0 

6 

6.00* 

4 th 

0 

1 

n.s. 

Totals 

2 

9 

4.46* 


*p<. 05, **p<.01 


No significant differences between the two forums for levels of all three types of presence during 
the first month of the semester were found. During the second month, for nine of the ten 
categories, no significant differences were found; interestingly, for one category of cognitive 
presence (triggering event), the forum led by the instructor held in low esteem was significantly 
more active. One last item is tabulated (Table 6), the ratio of triggering events with all instances 
of cognitive presence. 
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Table 6 

The Ratio of Instructors ’ Triggering Events with Cognitive Presence 



Foruml 

(low esteem) 

Forum2 

(high esteem) 

Triggering events 

8 

3 

Total cognitive presence 

16 

30 

Ratio 

1 : 2 

1 : 10 


The overall picture that emerges from these data is that during the second half of the semester 
(months 3 and 4), the instructor held in low esteem became nearly dysfunctional in her forum. 
Over this interval, she posted three times only (3 instances of social presence/open 
communication). This finding may indeed be the primary causal factor that led to the near 
cessation of active participation. The highly esteemed instructor was especially active from 
semester midpoint to semester end; she more than doubled her active participation in both 
teaching presence (especially discourse and instruction) and social presence (all three categories). 

In order to further explore active participation in the forums, we next investigated the teaching, 
social, and cognitive presence engaged in by students. No significant differences were found 
between forums for students’ teaching presence and cognitive presence. A highly significant 
difference was found (f - 21.29, p < .001) for students’ social presence: It was higher in the 
forum led by the highly esteemed instructor. Table 7 presents these data. 

Table 7 

Social Presence: Students’ Postings Only 


Categories 

Months 

Foruml 

(low esteem) 

Forum2 

(high esteem) 

x 2 

Affective 

1 st 

3 

1 

n.s. 

IP 

7 

0 

7.00** 


2 

11 

6.23* 

4 th 

0 

3 

n.s. 

Totals 

12 

15 

n.s. 

Open 

communication 

1 st 

5 

7 

n.s. 

IP 

8 

5 

n.s. 


2 

18 

12.80*** 

4 th 

0 

8 

8.00** 

Totals 

15 

38 

9.98** 

Cohesion 

1 st 

9 

9 

n.s. 

IP 

11 

5 

n.s. 

“P 

2 

31 

29.12*** 
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4 th 

1 

10 

7.36** 

Totals 

23 

55 

13.13*** 


* p< . 05 , **p<. 001 , ***/?<. 001 


The final parameter investigated was instructor response rates. These were determined from the 
course log. Response rates measure the time interval between students’ postings and instructors’ 
responses. The median response time for the highly esteemed instructor was 3.25 hours (N = 53) 
while the median response time for the instructor held in low esteem was 26.30 hours (N = 20). 
The difference in response time is profound. Furthermore, to the detriment of the lowly esteemed 
instructor, an additional six students’ postings (23% of the total messages posted) remained 
unanswered; these were not included in the median statistic. Unanswered postings began to occur 
in the third month. One student, who posted an unanswered query, wrote: “Why isn't anyone 
answering? Is anyone here? Should 1 turn the lights off?” Figure 2 shows the scatter plot for each 
instructor’s response time. 



Figure 2. Instructors’ response times (in hours). 

Discussion 

This ad hoc analysis, very carefully controlled in retrospect, identified the roles played by two 
online instructors (one held in high esteem and the other held in low esteem) and described their 
impact on two graduate level course forums wherein participation was voluntary. Based on 
theory and empirical findings, it was hypothesized that in a forum led by a highly rated instructor, 
as opposed to a forum led by an instructor held in low esteem, the following would occur: 

1 . Active and passive participation would be significantly higher. 

2. Social presence would be significantly higher. 

3. Teaching presence would be significantly higher. 

4. Cognitive presence would be significantly higher. 

5. Instructor response time would be significantly shorter. 
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All hypotheses, except for hypothesis 4, were supported clearly and unequivocally. In order to 
understand the implications of these findings, we next analyze active participation in the forum as 
a function of hypotheses 2-5. We begin by attempting to explain the unsupported hypothesis, 
namely that cognitive presence will be significantly higher in the forum led by the highly 
esteemed instructor. We asked ourselves why this did not happen and proceeded in two 
directions: (1) the need for defining specific goals or tasks for sustaining student engagement, and 
(2) perceived course difficulty. 


The Equivalent Sparsity of Cognitive Presence in Both Forums 

The emergence of cognitive presence in an online learning environment depends on many factors. 
One factor deemed significant by Garrison and Arbaugh (2007) is what they call the “progressive 
development of inquiry” (p. 1 62). They found that cognitive presence may be defined in terms of 
a “cyclical process of practical inquiry, where participants move deliberately from understanding 
the problem or issue through to exploration, integration and application” (p.162). Research has 
shown that this is oftentimes difficult to achieve (e.g., Luebeck & Bice, 2005; Vaughan & 
Garrison, 2005). 

In our study, specific tasks, tailored especially to the needs of the course forum, were not pre- 
prepared. The forum was more open-ended regarding the ways by which subject-matter oriented 
topics were discussed. This, however, does not preclude the progressive development of cognitive 
presence spontaneously, as a function of the instructor's teaching presence, especially “facilitating 
discourse.” We now analyze the extent of cognitive presence in both forums. 

On the one hand, during the first two months of the semester, the instructor held in low esteem 
initiated eight triggering events. This would indicate that she was indeed motivated to provide her 
students with learning opportunities. These initiatives, however, met with lukewarm response and 
petered out rapidly. The ratio of triggering events and total instances of cognitive presence was a 
mere 1 : 2 (Table 6). The number of threads for her forum (a measure of more extensive dialogue) 
was a mere 18 (Table 1). For this instructor, the lack of specific, progressively structured inquiry 
tasks and/or the lack of facilitation skills (teaching presence/facilitating discourse) may have 
contributed to the relatively limited occurrences of cognitive presence. 

On the other hand, for the instructor held in high esteem, the ratio of triggering events and total 
instances of cognitive presence was 1:10 (Table 6). The number of threads in her forum was 38 
(Table 1). This would indicate that for the highly esteemed instructor, the lack of specific pre- 
planned inquiry tasks did not apparently restrict achieving high levels of integration and 
resolution by way of teaching presence/facilitating discourse. In other words, we wish to reaffirm 
what we cited above: teaching presence, especially the category “facilitating discourse,” seems to 
play a highly significant role in achieving and sustaining cognitive presence (i.e., learning). 

The Open University questionnaire also elicits from students their evaluation of “course 
difficulty.” This course was rated as “average” (3 on a scale of 1-5). This non-difficulty may, in 
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part, also account for the general lack of cognitive presence found in both forums, despite the fact 
that the explicit goal of both forums was to provide a place to learn. Research has shown that 
students turned to interpersonal dialogue primarily when they couldn’t cope with conceptual 
difficulties or with difficult tutor assignments (Gorsky, Caspi, & Tuvi-Arad, 2004; Gorsky, Caspi, 
& Trumper, 2004, 2006; Gorsky, Caspi, & Smidt, 2007). Given the absence of such difficulties 
and that participation in the forum was non-mandatory, students seemed to have studied on their 
own. 

In other words, there are reasonable, theory-based explanations for the relative lack of cognitive 
presence found in both forums. This implies that something else accounted for the extreme 
satisfaction and dissatisfaction experienced by students in the two forums. The something else 
may be the two exceptional events that occurred during the third month: The instructor held in 
low esteem became nearly dysfunctional, while the highly esteemed instructor exhibited very 
high teacher presence and social presence (see Table 3 and 4). 


The Impact of Teaching Presence, Social Presence, and Instructor 
Response Time 

We found highly significant relationships between levels of social presence and teaching 
presence, on the one hand, and students’ active and passive participation in the forum and their 
satisfaction with it, on the other hand. These findings clearly support previous ones for both 
social presence (Richardson & Swan, 2002; Steinweg et al., 2006; Weaver & Albion, 2005) and 
for teaching presence (Garrison & Cleveland-Innes, 2005; Meyer, 2003; Murphy, 2004; Pawan et 
al., 2003; Shea et al., 2004; Swan, 2002; Swan & Shih, 2005; Vamhagen et al., 2005; Vaughan, 
2004; Wu & Hiltz, 2004). In addition, findings concerning instructor response time (frequent 
instructor responses correlate with students’ active and passive participation in the forum) are in 
accord with previous research (Kearsley, 2000; Northrup, 2002; Simonson et al., 2000; Swan, 
2001). In other words, these findings corroborate previous research. 

Moving beyond these studies, we wish to suggest a tentative two-tier model of causality, based on 
recently reported research findings. The model appears in Figure 3. 
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Figure 3. Factors affecting participation and satisfaction in academic forums. 

The first tier is based on the well-documented relationship between instructor response time, on 
the one hand, and student participation and satisfaction, on the other hand. The second tier of the 
model includes the mediating variable “students’ perceived learning.” Caspi and Blau (2008) and 
Swan and Richardson (2003) found that students’ perceived social presence in academic forums 
correlates with their perceived learning. This correlation points to the tentative conclusion that 
social presence affords learning by setting a convenient climate. In a similar vein, Shea, Pickett, 
and Pelt (2003) found that students’ perceived teacher presence also correlates with perceived 
learning as well as with students’ satisfaction with the forum. This correlation points to the 
tentative conclusion that teaching presence affords learning by setting a convenient climate. In 
our study, we found that very high versus very low instructor teaching presence and social 
presence during the third month affected students’ social presence, which, in turn, affected their 
active and passive participation as well as their sense of satisfaction. Based on these findings and 
on the three previous ones cited (Caspi & Blau, 2008; Shea et al., 2003; Swan & Richardson, 
2003), we suggest that students’ perceived learning in course forums has a significant impact on 
their participation (both active and passive) and satisfaction. In other words, “students’ perceived 
learning” may be a mediating variable as illustrated in Figure 3. This hypothesis, based on a 
combination of findings from different studies is, at best, speculative. Further research is needed 
to explore this explanation. 


Suggestions for Effective Online Teaching 

We suggest that teaching presence, social presence, and instructor response time appear to be 
important factors in the ratings attained by instructors under the very specific conditions 
described in this study. Table 8 compares findings for these factors along two dimensions: 
between-forum ratios and within-forum distributions expressed as percentages. These quantities 
may aid in evaluating the quality of academic forums that are similar to the ones described in this 
study. 
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Table 8 

Suggestions for Best and Worst Practice 


Criteria 

Best practice 

Worst practice 

Ratio* 

instances 

instances 


instances 

% of total 

instances 

% of total 

Instructor postings 

53 

42.4% 

20 

36.4% 

2.5 : 1 

Student postings 

72 

57.6% 

35 

63.6% 

2 : 1 

Totals 

125 

100% 

55 

100% 

2.5 : 1 

Teaching presence (totals) 

73 

19.9% 

43 

24.4% 

2 : 1 

Social presence (totals) 

236 

64.3% 

78 

44.3% 

3 : 1 

Cognitive presence (totals) 

58 

15.8% 

55 

31.3% 

1 : 1 

Totals 

367 

100% 

176 

100% 

2 : 1 

Teaching presence (instructor) 

52 

24.8% 

17 

22.7% 

3 : 1 

Social presence (instructor) 

128 

60.9% 

28 

37.3% 

5.5 : 1 

Cognitive presence (instructor) 

30 

14.3% 

30 

40.0% 

1 : 1 

Totals 

210 

100% 

75 

100% 

3 : 1 

Teaching presence (students) 

26 

15.0% 

21 

20.4% 

1 : 1 

Social presence (students) 

108 

62.4% 

53 

51.4% 

2 : 1 

Cognitive presence (students) 

39 

22.6% 

29 

28.2% 

1 : 1 

Totals 

173 

100% 

103 

100% 

2 : 1 

Threads 

38 


18 


2 : 1 

Instructor response time (median) 

3.25 hrs. 


26.50 hrs. 


1 : 8 


* rounded to nearest half integer 


These tentative reference points for best and worst practice are for illustrative purposes only. A 
great deal of additional research is required in order to define the attributes of these forums in 
terms of academic discipline, learning environment (blended), perceived course difficulty, 
communication type (asynchronous), duration, participation (non-mandatory), group size, etc. 
Such a classification is necessary in order to achieve a standard basis for comparison and 
reasonable confidence levels. However, the table is suggestive of the eventual possibility of 
having an “objective” tool for evaluating the quality of a given forum. 
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